Level of evidence of clinical neurosurgery research in Saudi Arabia

Good quality clinical research is fundamental for the practice of evidence-based medicine (EBM). Level of evidence (LOE) is a tool used to assess the quality and design of clinical research. It is generally accepted that publications with a high LOE are likely to be more valid and have greater impact on clinical practice. In recent years, there has been a growing interest in the assessment of the LOE in publications from the Kingdom of Saudi Arabia (KSA),1,2 in major specialty journals,3,4 and from different countries.5 Up to date information on the quality of neurosurgery research in KSA is still lacking. This study aimed at evaluating the LOE in Saudi neurosurgical publications, and comparing this with the international literature, and with KSA publications from other specialties.

G ood quality clinical research is fundamental for the practice of evidence-based medicine (EBM). Level of evidence (LOE) is a tool used to assess the quality and design of clinical research. It is generally accepted that publications with a high LOE are likely to be more valid and have greater impact on clinical practice. In recent years, there has been a growing interest in the assessment of the LOE in publications from the Kingdom of Saudi Arabia (KSA), 1,2 in major specialty journals, 3,4 and from different countries. 5 Up to date information on the quality of neurosurgery research in KSA is still lacking. This study aimed at evaluating the LOE in Saudi neurosurgical publications, and comparing this with the international literature, and with KSA publications from other specialties.
This study was carried out at the King Khalid National Guard Hospital (KKNGH), Jeddah, KSA between November and December 2013. It was a review based on routinely available open access data; hence, did not require ethical approval by KKNGH. The authors compiled a list of 73 consultant neurosurgeons that worked in KSA over the last 24 years by means of records from the Saudi Association of Neurological Surgery (SANS) membership lists and previous meetings programs. Using the name of each neurosurgeon, "PubMed," and "Google Scholar" searches were carried out to identify all KSA articles published between 1990-2013. The inclusion criteria were clinical neurosurgery and neurosciences articles published in English in any journal, where the name of at least one KSA neurosurgical center was represented in the authorship. Articles that reported basic research, laboratory work, and letters to editors were excluded.
Using the full publication, the first 2 authors assessed every article independently and discrepancies were resolved by discussion. Every study was ranked according to its LOE using Oxford's LOE Scale. 6 The LOE in the articles was evaluated and compared with respect to: the year of publication, whether during 1990-2000 or 2001-2013, the presence or absence of international collaboration, the journal's category, whether neuroscience (including neurosurgery) or other medical, the journal's impact factor (IF) whether equal and higher or lower than one, the research topic, whether related to neurosurgery or to other neurosciences and the article's citation numbers whether equal and higher or lower than 10. The findings were also compared with the LOE in Saudi orthopedics and plastic surgery publications, 1,2 as well as the LOE in spine 3 and neurosurgery 4 publications in major international journals, and in articles published by Iranian neurosurgeons. 5 GraphPad QuickCalcs (GraphPad Software, La Jolla, CA, USA) was used for the statistical analysis. A kappa value was calculated to estimate the level of agreement between the reviewers. The various pairs of LOE findings were examined statistically by comparing the differences in their LOE means and 95% confidence intervals (CI) using unpaired t test. A p<0.05 was considered statistically significant.
As a result of the exclusion of letters to editors and bench research there were no articles with LOE V in our study, which is comparable to the other Saudi reports. 1,2 Case reports accounted for 47.5% of articles, which is higher than the 13-41.9% quoted by others. [1][2][3][4][5] Commonly case reports are not included in the LOE grading because many believe that they are weak in assessing the efficacy of treatment. 3,4 We included them because they represented a substantial portion of the total KSA neurosurgery research output. We observed a significantly higher LOE in articles that were published during 2001-2013 compared with those that were published from 1990-2000. Such a finding, not observed in the Saudi literature, 1,2 is encouraging and may reflect increasing awareness of EBM by Saudi neurosurgeons. Two hospitals in Riyadh (KKUH and KFSHRC) accounted for 61% of the total publications. This reflects an imbalance between various KSA neurosurgical centers in their research contribution, which may be related to differences in experience, facilities, and clinical load. Alimi et al 5 reported a larger contribution to research by Iranian neurosurgeons working in universities and in major cities. Collaboration with international centers had a positive influence on the LOE and should be encouraged.
Neurosciences (Riyadh) was the most frequently used journal, and most of the articles were published in non-neurosurgical journals. Only 26% were published in Saudi Journals, which is lower than what others have cited for publications in local journals. 1,5 The articles were published in 99 journals and despite most of these journals having acceptable IFs, their large number is a signal that the KSA neurosurgical research needs to be more focused. We found that neither the journal's category nor its IF affected the LOE. Amiri et al 3 reported a significant variation in the LOE between articles published in 5 major spine journals that were consistent with their IFs. Rothoerl et al 4 on the other hand, found comparable LOE among articles published in 3 major neurosurgical journals that had different IFs. The most common research topics in KSA clinical neurosurgery publications were tumor and pediatric neurosurgery. Infection was well represented at 10.1%, and trauma at 7.7% was under-represented compared with others. 5 Neurosciences topics (non-neurosurgical) accounted for 16.2% of publications and were associated with a significantly higher LOE. This would indicate a positive aspect to research collaboration between neurosurgeons and their neurosciences colleagues. The median citation numbers of the articles was 11, and we found no significant difference in the LOE between articles that had citation numbers equal and higher or lower than 10. It is recognized that at times articles reporting case series and retrospective studies are the only source of data available so they become well cited. 3 Comparison of our findings with the LOE in publications from other KSA surgical specialties ( Table 1) showed that the neurosurgical LOE results were comparable to plastic surgery, 2 (4) www.neurosciencesjournal.org lower than orthopedic surgery, 1 despite the LOE in the latter 2 reported as comparable. 2 In addition, our LOE was significantly lower when compared with publications in the international neurosurgical literature, 4 in the international spine literature, 3 and in publications by Iranian neurosurgeons. 5 The LOE of clinical neurosurgery research in KSA over 24 years was low (level IV 91%, and prospective studies 6.4%). It is accepted that not all clinical research can be constructed in the method required for it to be classified as high LOE. [1][2][3] Nevertheless, it would be reasonable to assume that the majority of KSA neurosurgeons find the execution of high LOE clinical research difficult. The reasons are multifactorial, and include lack of experience, time, interest as well as limited logistic and financial support. In addition, the absence of qualified Saudi neurosurgeons holding research degrees as well as the dilution of the clinical load due to the non-regionalization of the service can be considered as barriers to conducting quality neurosurgical research in KSA.
There are limitations in the interpretation of our results. Some publications may have been missed due to search errors as a result of the misspelling of names and because the publications were in journals that are not indexed in "PubMed" and "Google Scholar". We believe this was minimized by searching in 2 databases. Relevant studies were unlikely to be missed as most articles had more than one Saudi author.
In conclusion, this study demonstrates a relatively low LOE of clinical neurosurgery in KSA compared with the literature. The LOE was significantly higher over the last decade, in the presence of international collaboration, and when the research topic was related to other neurosciences. The practice of EBM should be encouraged and local journals should consider the LOE of articles during the review process. More academic units should be established in KSA, and neurosurgeons *Including neurosurgery journals, †Excluding neurosurgery topics, ‡Only 481 articles were scored. IF -impact factor, Sig -Significant, NS -not significant, Ortho -orthopedics, Neuro -Neurosurgery